Neighbor management module for use in video encoding and methods for use therewith

ABSTRACT

A motion compensation module can be used in a video encoder for encoding a video input signal. The motion compensation module includes a neighbor management module that generates and stores neighbor data for at least one macroblock of the plurality of macroblocks for retrieval for retrieval by at least one of a motion search module, a motion refinement module, a direct mode module, and an intra-prediction module, when operating on at least one neighboring macroblock of the plurality of macroblocks.

TECHNICAL FIELD OF THE INVENTION

The present invention relates to filtering and encoding used in devicessuch as video encoders/codecs.

DESCRIPTION OF RELATED ART

Video encoding has become an important issue for modern video processingdevices. Robust encoding algorithms allow video signals to betransmitted with reduced bandwidth and stored in less memory. However,the accuracy of these encoding methods face the scrutiny of users thatare becoming accustomed to greater resolution and higher picturequality. Standards have been promulgated for many encoding methodsincluding the H.264 standard that is also referred to as MPEG-4, part 10or Advanced Video Coding, (AVC). While this standard sets forth manypowerful techniques, further improvements are possible to improve theperformance and speed of implementation of such methods.

Further limitations and disadvantages of conventional and traditionalapproaches will become apparent to one of ordinary skill in the artthrough comparison of such systems with the present invention.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIGS. 1-3 present pictorial diagram representations of a various videoprocessing devices in accordance with embodiments of the presentinvention.

FIG. 4 presents a block diagram representation of a video processingdevice 125 in accordance with an embodiment of the present invention.

FIG. 5 presents a block diagram representation of a video encoder 102that includes neighbor management module 218 in accordance with anembodiment of the present invention.

FIG. 6 presents a graphical representation of the relationship betweenexample top frame and bottom frame macroblocks (250, 252) and exampletop field and bottom field macroblocks (254, 256) in accordance with anembodiment of the present invention.

FIG. 7 presents a graphical representation that shows example macroblockpartitioning in accordance with an embodiment of the present invention.

FIG. 8 presents a graphical representation of a plurality of macroblocksof a video input signal that shows an example of the neighboringmacroblocks used in motion compensation or encoding of a particularmacroblock.

FIG. 9 presents a flowchart representation of a method in accordancewith an embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION INCLUDING THE PRESENTLY PREFERREDEMBODIMENTS

FIGS. 1-3 present pictorial diagram representations of a various videoprocessing devices in accordance with embodiments of the presentinvention. In particular, set top box 10 with built-in digital videorecorder functionality or a stand alone digital video recorder, computer20 and portable computer 30 illustrate electronic devices thatincorporate a video processing device 125 that includes one or morefeatures or functions of the present invention. While these particulardevices are illustrated, video processing device 125 includes any devicethat is capable of encoding video content in accordance with the methodsand systems described in conjunction with FIGS. 4-9 and the appendedclaims.

FIG. 4 presents a block diagram representation of a video processingdevice 125 in accordance with an embodiment of the present invention. Inparticular, video processing device 125 includes a receiving module 100,such as a television receiver, cable television receiver, satellitebroadcast receiver, broadband modem, 3G transceiver or other informationreceiver or transceiver that is capable of receiving a received signal98 and extracting one or more video signals 110 via time divisiondemultiplexing, frequency division demultiplexing or otherdemultiplexing technique. Video encoding module 102 is coupled to thereceiving module 100 to encode or transcode the video signal in a formatcorresponding to video display device 104.

In an embodiment of the present invention, the received signal 98 is abroadcast video signal, such as a television signal, high definitiontelevisions signal, enhanced high definition television signal or otherbroadcast video signal that has been transmitted over a wireless medium,either directly or through one or more satellites or other relaystations or through a cable network, optical network or othertransmission network. In addition, received signal 98 can be generatedfrom a stored video file, played back from a recording medium such as amagnetic tape, magnetic disk or optical disk, and can include astreaming video signal that is transmitted over a public or privatenetwork such as a local area network, wide area network, metropolitanarea network or the Internet.

Video signal 110 can include an analog video signal that is formatted inany of a number of video formats including National Television SystemsCommittee (NTSC), Phase Alternating Line (PAL) or Sequentiel CouleurAvec Memoire (SECAM). Processed video signal includes 112 a digitalvideo codec standard such as H.264, MPEG-4 Part 10 Advanced Video Coding(AVC) or other digital format such as a Motion Picture Experts Group(MPEG) format (such as MPEG1, MPEG2 or MPEG4), Quicktime format, RealMedia format, Windows Media Video (WMV) or Audio Video Interleave (AVI),or another digital video format, either standard or proprietary.

Video display devices 104 can include a television, monitor, computer,handheld device or other video display device that creates an opticalimage stream either directly or indirectly, such as by projection, basedon decoding the processed video signal 112 either as a streaming videosignal or by playback of a stored digital video file.

Video encoder 102 includes a motion compensation module 150 thatoperates in accordance with the present invention and, in particular,includes many optional functions and features described in conjunctionwith FIGS. 5-9 that follow.

FIG. 5 presents a block diagram representation of a video encoder 102that includes neighbor management module 218 in accordance with anembodiment of the present invention. In particular, video encoder 102operates in accordance with many of the functions and features of theH.264 standard, the MPEG-4 standard, VC-1 (SMPTE standard 421M) or otherstandard, to encode a video input signal 110 that is converted to adigital format via a signal interface 198.

The video encoder 102 includes a processing module 200 that can beimplemented using a single processing device or a plurality ofprocessing devices. Such a processing device may be a microprocessor,co-processors, a micro-controller, digital signal processor,microcomputer, central processing unit, field programmable gate array,programmable logic device, state machine, logic circuitry, analogcircuitry, digital circuitry, and/or any device that manipulates signals(analog and/or digital) based on operational instructions that arestored in a memory, such as memory module 202. Memory module 202 may bea single memory device or a plurality of memory devices. Such a memorydevice can include a hard disk drive or other disk drive, read-onlymemory, random access memory, volatile memory, non-volatile memory,static memory, dynamic memory, flash memory, cache memory, and/or anydevice that stores digital information. Note that when the processingmodule implements one or more of its functions via a state machine,analog circuitry, digital circuitry, and/or logic circuitry, the memorystoring the corresponding operational instructions may be embeddedwithin, or external to, the circuitry comprising the state machine,analog circuitry, digital circuitry, and/or logic circuitry.

Processing module 200, and memory module 202 are coupled, via bus 220,to the signal interface 198 and a plurality of other modules, such asmotion search module 204, motion refinement module 206, direct modemodule 208, intra-prediction module 210, mode decision module 212,reconstruction module 214, entropy coding module 216, neighbormanagement module 218, forward transform and quantization module 220 anddeblocking filter module 222. The modules of video encoder 102 can beimplemented in software, firmware or hardware, depending on theparticular implementation of processing module 200. It should also benoted that the software implementations of the present invention can bestored on a tangible storage medium such as a magnetic or optical disk,read-only memory or random access memory and also be produced as anarticle of manufacture. While a particular bus architecture is shown,alternative architectures using direct connectivity between one or moremodules and/or additional busses can likewise be implemented inaccordance with the present invention.

Motion compensation module 150 includes a motion search module 204 thatprocesses pictures from the video input signal 110 based on asegmentation into macroblocks of pixel values, such as of 16 pixels by16 pixels size, from the columns and rows of a frame and/or field of thevideo input signal 110. In an embodiment of the present invention, themotion search module determines, for each macroblock or macroblock pairof a field and/or frame of the video signal one or more motion vectors(depending on the partitioning of the macroblock into subblocks asdescribed further in conjunction with FIG. 7) that represents thedisplacement of the macroblock (or subblock) from a reference frame orreference field of the video signal to a current frame or field. Inoperation, the motion search module operates within a search range tolocate a macroblock (or subblock) in the current frame or field to aninteger pixel level accuracy such as to a resolution of 1-pixel.Candidate locations are evaluated based on a cost formulation todetermine the location and corresponding motion vector that have a mostfavorable (such as lowest) cost.

In an embodiment of the present invention, a cost formulation is basedon the sum of the Sum of Absolute Difference (SAD) between the referencemacroblock and candidate macroblock pixel values and a weighted rateterm that represents the number of bits required to be spent on codingthe difference between the candidate motion vector and either apredicted motion vector (PMV) that is based on the neighboringmacroblock to the right of the current macroblock and on motion vectorsfrom neighboring current macroblocks of a prior row of the video inputsignal or an estimated predicted motion vector that is determined basedon motion vectors from neighboring current macroblocks of a prior row ofthe video input signal. In addition, the cost calculation avoids the useof neighboring subblocks within the current macroblock. In this fashion,motion search module 204 is able to operate on a macroblock tocontemporaneously determine the motion search motion vector for eachsubblock of the macroblock.

A motion refinement module 206 generates a refined motion vector foreach macroblock of the plurality of macroblocks, based on the motionsearch motion vector. In an embodiment of the present invention, themotion refinement module determines, for each macroblock or macroblockpair of a field and/or frame of the video input signal 110 a refinedmotion vector that represents the displacement of the macroblock from areference frame or reference field of the video signal to a currentframe or field. In operation, the motion refinement module refines thelocation of the macroblock in the current frame or field to a greaterpixel level accuracy such as to a resolution of ¼-pixel. Candidatelocations are also evaluated based on a cost formulation to determinethe location and refined motion vector that have a most favorable (suchas lowest) cost. As in the case with the motion search module, a costformulation is based on the a sum of the Sum of Absolute Difference(SAD) between the reference macroblock and candidate macroblock pixelvalues and a weighted rate term that represents the number of bitsrequired to be spent on coding the difference between the candidatemotion vector and either a predicted motion vector (PMV) that is basedon the neighboring macroblock to the right of the current macroblock andon motion vectors from neighboring current macroblocks of a prior row ofthe video input signal or an estimated predicted motion vector that isdetermined based on motion vectors from neighboring current macroblocksof a prior row of the video input signal. In addition, the costcalculation avoids the use of neighboring subblocks within the currentmacroblock. In this fashion, motion refinement module 206 is able tooperate on a macroblock to contemporaneously determine the motion searchmotion vector for each subblock of the macroblock.

It should be noted that when estimated predicted motion vectors are usedthe cost formulation avoids the use of motion vectors from the currentrow and both the motion search module 204 and the motion refinementmodule 206 can operate in a pipelined fashion and in parallel on anentire row of video input signal 110, to contemporaneously determine therefined motion vector for each macroblock in the row.

A direct mode module 208 generates a direct mode motion vector for eachmacroblock of the plurality of macroblocks, based on a plurality ofmacroblocks that neighbor the macroblock of pixels. In an embodiment ofthe present invention, the direct mode module 208 operates in a fashionsuch as defined by the H.264 standard to determine the direct modemotion vector and the cost associated with the direct mode motionvector.

While the prior modules have focused on inter-prediction of the motionvector, intra-prediction module 210 generates a best intra predictionmode for each macroblock of the plurality of macroblocks. In particular,intra-prediction module 210 operates in a fashion such as defined by theH.264 standard to evaluate a plurality of intra prediction modes, basedon motion vectors determined from neighboring macroblocks to determinethe best intra prediction mode and the associated cost.

A mode decision module 212 determines a final motion vector for eachmacroblock of the plurality of macroblocks based on costs associatedwith the refined motion vector, the direct mode motion vector, and thebest intra prediction mode, and in particular, the method that yieldsthe most favorable (lowest) cost, or otherwise an acceptable cost. Areconstruction module 214 generates residual luma and chroma pixelvalues corresponding to the final motion vector for each macroblock ofthe plurality of macroblocks.

An forward transform and quantization module 220 of video encoder 102generates processed video signal 112 by transforming coding andquantizing the residual pixel values into quantized transformedcoefficients that can be further coded, such as by entropy coding inentropy coding module 216, filtered by de-blocking filter module 222 andtransmitted and/or stored as the processed video signal 112.

As discussed above, many of the modules of motion compensation module150 operate based on motion vectors determined for neighboringmacroblocks. Neighbor management module 218 generates and storesneighbor data for at least one macroblock of the plurality ofmacroblocks for retrieval by at least one of the motion search module204, the motion refinement module 206, the direct mode module 208,intra-prediction module 210, entropy coding module 216 and deblockingfilter module 222, when operating on at least one neighboring macroblockof the plurality of macroblocks. As the motion vector (or the pluralityof motion vectors in the case of macroblock partitioning, discussedfurther in conjunction with FIGS. 7 and 8) and the other encoding dataare finalized, neighboring data is stored for use in the processing ofneighboring macroblocks that have yet to be processed, yet that willrequire the use of such data. In addition, neighboring data is alsostored for the processing of future pictures, such as future framesand/or fields of video input signal 110.

In an embodiment of the present invention, a data structure, such as alinked list, array or one or more registers are used to associate andstore neighbor data for each macroblock. Neighbor data includes motionvectors, reference indices, quantization parameters, coded-blockpatterns, macroblock types, intra/inter prediction module typesneighboring pixel values and or other data from neighboring macroblocksand/or subblocks used to by one or more of the modules or procedures ofthe present invention to calculate results for a current macroblock. Forexample, in order to determine the predicated motion vector for themotion search module 204 and motion refinement module 206, both themotion vectors and reference index of neighbors are required. Inaddition to these data, the direct mode module 208 requires the motionvectors of the co-located macroblock of previous reference pictures. Thedeblocking filter module 222 operates according to a set of filteringstrengths determined by using the neighbors' motion vectors,quantization parameters, reference index, and coded-block-patterns, etc.For entropy coding in entropy coding module 216, the motion vectordifferences (MVD), macroblock types, quantization parameter delta, interpredication type, etc. are required.

Consider the example where a particular macroblock MB(x,y) requiresneighbor data from macroblocks MB(x−1, y−1), MB(x, y−1), MB (x+1,y−1)and MB(x−1,y). In prior art codecs, the preparation of the neighbor dataneeds to calculate the location of the relevant neighbor sub-blocks.However, the calculation is not as straightforward as it was inconventional video coding standards. For example, in H.264 coding, thesupport of multiple partition types make the size and shape for thesubblocks vary significantly. Furthermore, the support of the macroblockadaptive frame and field (MBAFF) coding allows the macroblocks to beeither in frame or in field mode. For each mode, one neighbor derivationmethod is defined in H.264. So the calculation needs to consider eachmode accordingly. In addition, in order to get all of the neighbor datarequired, the derivation needs to be invoked four times since there arefour neighbors involved—MB(x−1, y−1), MB(x, y−1), MB(x+1, y−1), andMB(x−1, y). So the encoding of the current macroblock MB(x, y) cannotstart until the location of the four neighbors has been determined andtheir data have been fetched from memory.

The present invention avoids the above problems. In particular when eachmacroblock is processed and final motion vectors and encoded data aredetermined, neighbor data is stored in data structures for eachneighboring macroblock that will need this data. Since the neighbor datais prepared in advance, the current macroblock MB(x,y) can start rightaway when it is ready to be processed. The burden of pinpointingneighbors is virtually re-allocated to its preceding macroblocks. Theencoding of macroblocks can be therefore be more streamline and faster.In other words, when the final motion vectors are determined forMB(x−1,y−1), neighbor data is stored for each neighboring macroblockthat is yet to be processed, including MB(x,y) and also otherneighboring macroblocks such as MB(x, y−1), MB(x−2,y) MB(x−1,y).Similarly, when the final motion vectors are determined for MB(x,y−1),MB (x+1,y−1) and MB(x−1,y) neighbor data is stored for each neighboringmacroblock corresponding to each of these macroblocks that are yet to beprocessed, including MB(x,y). In this fashion, when MB(x,y) is ready tobe processed, the neighbor data is already stored in a data structurethat corresponds to this macroblock for fast retrieval.

The motion compensation can then proceed using the retrieved data. Inparticular, the motion search module 204 and/or the motion refinementmodule, can generate at least one predicted motion vector (such as astandard PMV or estimated predicted motion vector) for each macroblockof the plurality of macroblocks using retrieved neighbor data. Further,the direct mode module 208 can generate at least one direct mode motionvector for each macroblock of the plurality of macroblocks usingretrieved neighbor data and the intra-prediction module 210 cangenerates the best intra prediction mode for each macroblock of theplurality of macroblocks using retrieved neighbor data, and the codingmodule 216 can use retrieved neighbor data in entropy coding, each asset forth in the H.264 standard, the MPEG-4 standard, VC-1 (SMPTEstandard 421M) or by other standard or other means.

While not expressly shown, video encoder 102 can include a memory cache,a memory management module, a comb filter or other video filter, and/orother module to support the encoding of video input signal 110 intoprocessed video signal 112.

FIG. 6 presents a graphical representation of the relationship betweenexample top frame and bottom frame macroblocks (250, 252) and exampletop field and bottom field macroblocks (254, 256) in accordance with anembodiment of the present invention. In this embodiment, motion searchmodule 204 generates a motion search motion vector for each macroblockof a plurality of macroblocks by contemporaneously evaluating amacroblock pair that includes a top frame macroblock 250 and bottomframe macroblock 252 from a frame of the video input signal 110 and atop field macroblock 254 and a bottom field macroblock 256 fromcorresponding fields of the video input signal 110.

Considering the example shown, each of the macroblocks are 16 pixels by16 pixels in size. Motion search is performed in full pixel resolution,or other resolution, either coarser or finer, by comparing a candidateframe macroblock pair of a current frame that includes top framemacroblock 250 and bottom frame macroblock 252 to the macroblock pair ofa reference frame. In addition, lines of a first parity (such as oddlines) from the candidate frame macroblock pair are grouped to form topfield macroblock 254. Similarly, lines of a second parity (such as evenlines) from the candidate frame macroblock pair are grouped to formbottom field macroblock 256. Motion search module 204 calculates a costassociated a plurality of lines, and generates a cost associated withthe top frame macroblock 250 based on a cost accumulated for a pluralityof top lines of the plurality of lines, generates a cost associated withthe bottom frame macroblock 252 based on a cost accumulated for aplurality of bottom lines of the plurality of lines, generates a costassociated with the top field macroblock 254 based on a cost accumulatedfor a plurality of first-parity lines of the plurality of lines comparedwith either a top or bottom field reference, and generates a costassociated with the bottom field macroblock 256 based on a costaccumulated for a plurality of second-parity lines of the plurality oflines, also based on either a top or bottom field reference. In thisfashion, six costs can be generated contemporaneously for the macroblockpair: top frame compared with top frame of the reference; bottom framecompared with the bottom frame of the reference; top field compared withtop field of the reference; bottom field compared with the bottom fieldof the reference; top field compared with bottom field of the reference;and bottom field compared with the top field of the reference.

Each of these costs can be generated based on the sum of the absolutedifferences (SAD) of the pixel values of the current frame or field withthe reference frame or field. The SADs can be calculatedcontemporaneously, in a single pass, based on the accumulation for eachline. The overall SAD for a particular macroblock (top or bottom, frameor field) can be determined by totaling the SADs for the lines that makeup that particular macroblock. Alternatively, the SADs can be calculatedin a single pass, based on the smaller segments such as 4×1 segmentsthat can be accumulated into subblocks, that in turn can be accumulatedinto overall macroblock totals. This alternative arrangementparticularly lends itself to motion search modules that operate based onthe partitioning of macroblocks into smaller subblocks, as will bediscussed further in conjunction with FIG. 7.

The motion search module 204 is particularly well adapted to operationin conjunction with macroblock adaptive frame and field processing.Frame mode costs for the current macroblock pair can be generated asdiscussed above. In addition, motion search module 204 optionallygenerates a field decision based on accumulated differences, such asSAD, between the current bottom field macroblock and a bottom fieldmacroblock reference, the current bottom field macroblock and a topfield macroblock reference, the current top field macroblock and thebottom field macroblock reference, and the current top field macroblockand the top field macroblock reference. The field decision includesdetermining which combination (top/top, bottom/bottom) or (top/bottom,bottom/top) yields a lower cost. Similarly, motion search module 204 canoptionally choose either frame mode or field mode for a particularmacroblock pair, based on whether the frame mode cost compares morefavorably (e.g. are lower) or less favorably (e.g. higher) to the fieldmode cost, based on the field mode decision. In addition, other modulesof motion compensation module 150 that operate on both frames and fieldcan operate can similarly operate.

In particular, the neighbor management module 218 generates neighbordata that includes frame below neighbor data for retrieval by aneighboring macroblock in a row below the at least one macroblock whenprocessing in frame mode and field below neighbor data for retrieval bythe neighboring macroblock in a row below the at least one macroblockwhen processing in field mode. In addition, the neighbor data includesframe right neighbor data for retrieval by a neighboring macroblock tothe right of the at least one macroblock when processing in field modeand field right neighbor data for retrieval by the neighboringmacroblock to the right of the at least one macroblock when processingin field mode. In this fashion, the motion search module and othermodules of motion compensation module 150 that operate using neighbordata and that can operate in either a frame or field mode can directlyaccess either the frame mode neighbor data for frame mode neighborsabove the macroblock of interest, the field mode neighbor data for fieldmode neighbors above the macroblock of interest, the frame mode neighbordata for the frame mode neighbor to the left of the macroblock ofinterest and/or the field mode neighbor data for the field mode neighborto the left of the macroblock of interest. As before, this informationis stored in the processing of the prior macroblocks, whether themacroblocks themselves were processed in frame or in field mode, and canbe accessed in the processing of the macroblock of interest by retrievaldirectly from memory and without a look-up table or further processing.

FIG. 7 presents a graphical representation of example partitionings of amacroblock of a video input signal into a plurality of subblocks. Inparticular, while the modules described in conjunction with FIG. 5 abovecan operate on macroblocks having a size such as 16 pixels×16 pixels,such as in accordance with the H.264 standard, macroblocks can bepartitioned into subblocks of smaller size, as small as 4 pixels on aside with the functions and features described in conjunction with themacroblocks applying to each subblock with individual pixel locationsindicated by dots. For example, motion search module 204 can generateseparate motion search motion vectors for each subblock of eachmacroblock, etc.

Macroblock 302 represents an example of partitioning into subblocks inaccordance with the H.264 standard. Macroblocks 300, 304 and 306represent examples of other possible partitioning into subblocks. Inparticular, macroblock 300 is a 16×16 macroblock that is partitionedinto an 8×16 subblock and two 8×8 subblocks. Macroblock 302 is a 16×16macroblock that is partitioned into three 8×8 subblocks and four 4×4subblocks. Macroblock 304 is a 16×16 macroblock that is partitioned intoan 8×16 subblock, an 8×8 subblock and two 4×8 subblocks. Macroblock 306is a 16×16 macroblock that is partitioned into an 8×8 subblock, three4×8 subblocks, two 8×4 subblocks, and two 4×4 subblocks. Thepartitioning of the macroblocks into smaller subblocks increases thecomplexity of the motion compensation by requiring various compensationmethods, such as the motion search to determine, not only the motionsearch motion vectors for each subblock, but the best motion vectorsover the set of all possible partitions of a particular macroblock. Theresult however can yield more accurate motion compensation and reducedcompression artifacts in the decoded video image.

FIG. 8 presents a graphical representation of a plurality of macroblocksof a video input signal that shows an example of the neighboringmacroblocks used in motion compensation or encoding of a particularmacroblock. Three macroblocks MB n−1, MB n and MB n+1 are show for threerows, row i−1, row i and row i+1 of a video input signal in either frameor field mode. The dots representing individual pixel locations havebeen omitted for clarity.

Consider for example, that video encoder 102 is operating on macroblockMB(n, i). Consider further, that the motion refinement module 206,motion search module 204, direct mode module 208, the intra-predictionmodule 210 and coding module 216 may need the final motion vectorsdetermined for 4×4 subblock D0 from MB(n−1, i−1), subblock B0 from MB(n,i−1), subblock C0 from MB(n+1, i−1), along with subblock A0 from MB(n−1,i). When MB(n−1, i−1) is processed, the motion vector for D0 is storedin a data structure associated with MB(n, i), along with the otherneighbor data for other neighbors such as MB(n, i−1), MB(n−2, i) andMB(n−1, i). When MB(n, i−1) is processed, the motion vector for B0 isstored in a data structure associated with MB(n, i) along with the otherneighbor data for other neighbors. When MB(n+1, i−1) is processed, themotion vector for C0 is stored in a data structure associated with MB(n,i) along with the other neighbor data for other neighbors. And whenMB(n−1, i) is processed, the motion vector for D0 is stored in a datastructure associated with MB(n, i) along with the other neighbor datafor other neighbors. In this fashion, when MB (n, i) is processed, anyof the necessary neighbor data can be easily retrieved from the datastructure.

While the above discussion relates to the processing in either frame offield mode, as discussed in conjunction with FIG. 6, both frame andfield mode neighbor data can be stored for later retrieval, as needed,in the processing of neighboring macroblocks. Further, while the abovediscussion focuses on individual macroblocks, neighbor data based on theprocessing or macroblock pairs can also be stored, with, for instance,neighbor data used by the bottom macroblock that is derived from the topmacroblock within the macroblock pair being generated directly in theprocessing of the macroblock pair.

FIG. 9 presents a flowchart representation of a method in accordancewith an embodiment of the present invention. In particular, a method ispresented for use in conjunction with one or more of the features andfunctions described in association with FIGS. 1-7. In step 400, one ormore motion search motion vectors are generated for each macroblock of aplurality of macroblocks. In step 402, a refined motion vector isgenerated for each macroblock of the plurality of macroblocks, based onthe one or more motion search motion vectors. In step 404, a direct modemotion vector is generated for each macroblock of the plurality ofmacroblocks, based on a plurality of macroblocks that neighbor themacroblock of pixels. In step 406, a best intra prediction mode isgenerated for each macroblock of the plurality of macroblocks.

In step 408, a final motion vector is determined for each macroblock ofthe plurality of macroblocks based on costs associated with the refinedmotion vector, the direct mode motion vector, and the best intraprediction mode. In step 410, residual pixel values are generatedcorresponding to the final motion vector for each macroblock of theplurality of macroblocks. In step 412, neighbor data is generated andstored for at least one macroblock of the plurality of macroblocks forretrieval by at least one of the steps of generating a motion searchmotion vector, generating a refined motion vector, generating a directmode motion vector, and generating a best intra prediction mode, whenoperating on at least one neighboring macroblock of the plurality ofmacroblocks.

In an embodiment of the present invention, steps 400, 402, 404 and/or406 operate in a macroblock adaptive frame and field mode and analyzeeach macroblock of a plurality of macroblocks based on macroblock pairsthat include a top frame macroblock and bottom frame macroblock from aframe of the video input signal and a top field macroblock and a bottomfield macroblock from a corresponding field of the video input signal.The neighbor data can include include frame below neighbor data forretrieval by a neighboring macroblock in a row below the at least onemacroblock when processing in frame mode and field below neighbor datafor retrieval by the neighboring macroblock in a row below the at leastone macroblock when processing in field mode. In addition, the neighbordata can include frame right neighbor data for retrieval by aneighboring macroblock to the right of the at least one macroblock whenprocessing in frame mode and field right neighbor data for retrieval bythe neighboring macroblock to the right of the at least one macroblockwhen processing in field mode.

In an embodiment, steps 400 and/or 402 generate at least one predictedmotion vector for each macroblock of the plurality of macroblocks usingretrieved neighbor data. Further, step 404 can generate at least onedirect mode motion vector for each macroblock of the plurality ofmacroblocks using retrieved neighbor data. Also, step 406 can generatethe best intra prediction mode for each macroblock of the plurality ofmacroblocks using retrieved neighbor data.

In preferred embodiments, the various circuit components are implementedusing 0.35 micron or smaller CMOS technology. Provided however thatother circuit technologies, both integrated or non-integrated, may beused within the broad scope of the present invention.

As one of ordinary skill in the art will appreciate, the term“substantially” or “approximately”, as may be used herein, provides anindustry-accepted tolerance to its corresponding term and/or relativitybetween items. Such an industry-accepted tolerance ranges from less thanone percent to twenty percent and corresponds to, but is not limited to,component values, integrated circuit process variations, temperaturevariations, rise and fall times, and/or thermal noise. Such relativitybetween items ranges from a difference of a few percent to magnitudedifferences. As one of ordinary skill in the art will furtherappreciate, the term “coupled”, as may be used herein, includes directcoupling and indirect coupling via another component, element, circuit,or module where, for indirect coupling, the intervening component,element, circuit, or module does not modify the information of a signalbut may adjust its current level, voltage level, and/or power level. Asone of ordinary skill in the art will also appreciate, inferred coupling(i.e., where one element is coupled to another element by inference)includes direct and indirect coupling between two elements in the samemanner as “coupled”. As one of ordinary skill in the art will furtherappreciate, the term “compares favorably”, as may be used herein,indicates that a comparison between two or more elements, items,signals, etc., provides a desired relationship. For example, when thedesired relationship is that signal 1 has a greater magnitude thansignal 2, a favorable comparison may be achieved when the magnitude ofsignal 1 is greater than that of signal 2 or when the magnitude ofsignal 2 is less than that of signal 1.

As the term module is used in the description of the various embodimentsof the present invention, a module includes a functional block that isimplemented in hardware, software, and/or firmware that performs one ormodule functions such as the processing of an input signal to produce anoutput signal. As used herein, a module may contain submodules thatthemselves are modules.

Thus, there has been described herein an apparatus and method, as wellas several embodiments including a preferred embodiment, forimplementing a video encoder and motion compensation module for usetherewith. Various embodiments of the present invention herein-describedhave features that distinguish the present invention from the prior art.

It will be apparent to those skilled in the art that the disclosedinvention may be modified in numerous ways and may assume manyembodiments other than the preferred forms specifically set out anddescribed above. Accordingly, it is intended by the appended claims tocover all modifications of the invention which fall within the truespirit and scope of the invention.

What is claimed is:
 1. A motion compensation module for use in a videoencoder for encoding a video input signal, the motion compensationmodule comprising: a motion search module, that generates a motionsearch motion vector for each macroblock of a plurality of macroblocks;a motion refinement module, coupled to the motion search module, thatgenerates a refined motion vector for each macroblock of the pluralityof macroblocks, based on the motion search motion vector; a direct modemodule, that generates a direct mode motion vector for each macroblockof the plurality of macroblocks, based on a plurality of macroblocksthat neighbor the macroblock of pixels; an intra-prediction module thatgenerates a best intra prediction mode for each macroblock of theplurality of macroblocks; a mode decision module, coupled to the motionrefinement module, the direct mode module and the prediction module,that determines a final motion vector for each macroblock of theplurality of macroblocks based on costs associated with the refinedmotion vector, the direct mode motion vector, and the best intraprediction mode; a reconstruction module, coupled to the mode decisionmodule, that generates residual pixel values corresponding to the finalmotion vector for each macroblock of the plurality of macroblocks; and aneighbor management module, coupled to the mode decision module, thatgenerates and stores neighbor data that includes motion vector dataassociated with one macroblock of the plurality of macroblocks in aplurality of data structures associated with each of a correspondingplurality of neighboring macroblocks for retrieval by at least one ofthe motion search module, the motion refinement module, and the directmode module when operating on at least one neighboring macroblock of theplurality of macroblocks to form a temporal prediction; wherein at leastone of the motion search module, the motion refinement module, thedirect mode module, the intra-prediction module, and the mode decisionmodule operates in a macroblock adaptive frame and field mode andanalyze each macroblock of a plurality of macroblocks based onmacroblock pairs that include a top frame macroblock and bottom framemacroblock from a frame of the video input signal and a top fieldmacroblock and a bottom field macroblock from a corresponding field ofthe video input signal.
 2. The motion compensation module of claim 1wherein neighbor data includes frame below neighbor data for retrievalby a neighboring macroblock in a row below the one macroblock whenprocessing in frame mode and field below neighbor data for retrieval bythe neighboring macroblock in a row below the one macroblock whenprocessing in field mode.
 3. The motion compensation module of claim 1wherein neighbor data includes frame right neighbor data for retrievalby a neighboring macroblock to the right of the one macroblock whenprocessing in frame mode and field right neighbor data for retrieval bythe neighboring macroblock to the right of the one macroblock whenprocessing in field mode.
 4. The motion compensation module of claim 1wherein at least one of the motion search module and the motionrefinement module, generate at least one predicted motion vector foreach macroblock of the plurality of macroblocks using retrieved neighbordata.
 5. The motion compensation module of claim 1 wherein the directmode module generates at least one direct mode motion vector for eachmacroblock of the plurality of macroblocks using retrieved neighbordata.
 6. The motion compensation module of claim 1 wherein theintra-prediction module generates the best intra prediction mode foreach macroblock of the plurality of macroblocks using retrieved neighbordata.
 7. A motion compensation module for use in a video encoder forencoding a video input signal, the motion compensation modulecomprising: a motion search module, that generates a motion searchmotion vector for each macroblock of a plurality of macroblocks; amotion refinement module, coupled to the motion search module, thatgenerates a refined motion vector for each macroblock of the pluralityof macroblocks, based on the motion search motion vector; a direct modemodule, that generates a direct mode motion vector for each macroblockof the plurality of macroblocks, based on a plurality of macroblocksthat neighbor the macroblock of pixels; an intra-prediction module thatgenerates a best intra prediction mode for each macroblock of theplurality of macroblocks; a mode decision module, coupled to the motionrefinement module, the direct mode module and the prediction module,that determines a final motion vector for each macroblock of theplurality of macroblocks based on costs associated with the refinedmotion vector, the direct mode motion vector, and the best intraprediction mode; a reconstruction module, coupled to the mode decisionmodule, that generates residual pixel values corresponding to the finalmotion vector for each macroblock of the plurality of macroblocks; and aneighbor management module, coupled to the mode decision module, thatgenerates and stores neighbor data that includes motion vector dataassociated with one macroblock of the plurality of macroblocks in aplurality of data structures associated with each of a correspondingplurality of neighboring macroblocks for retrieval by at least one ofthe motion search module, the motion refinement module, and the directmode module, when operating on at least one neighboring macroblock ofthe plurality of macroblocks to form a temporal prediction.
 8. Themotion compensation module of claim 7 operates in macroblock adaptiveframe and field mode and analyzes each macroblock of a plurality ofmacroblocks based on macroblock pairs that include a top framemacroblock and bottom frame macroblock from a frame of the video inputsignal and a top field macroblock and a bottom field macroblock from acorresponding field of the video input signal.
 9. The motioncompensation module of claim 7 wherein neighbor data includes framebelow neighbor data for retrieval by a neighboring macroblock in a rowbelow the one macroblock when processing in frame mode and field belowneighbor data for retrieval by the neighboring macroblock in a row belowthe one macroblock when processing in field mode.
 10. The motioncompensation module of claim 7 wherein neighbor data includes frameright neighbor data for retrieval by a neighboring macroblock to theright of the one macroblock when processing in frame mode and fieldright neighbor data for retrieval by the neighboring macroblock to theright of the one macroblock when processing in field mode.
 11. Themotion compensation module of claim 7 wherein at least one of the motionsearch module and the motion refinement module, generate at least onepredicted motion vector for each macroblock of the plurality ofmacroblocks using retrieved neighbor data.
 12. The motion compensationmodule of claim 7 wherein the direct mode module generates at least onedirect mode motion vector for each macroblock of the plurality ofmacroblocks using retrieved neighbor data.
 13. The motion compensationmodule of claim 7 wherein the intra-prediction module generates the bestintra prediction mode for each macroblock of the plurality ofmacroblocks using retrieved neighbor data.
 14. A method for use in avideo encoder for encoding a video input signal, the motion compensationmodule comprising: generating a motion search motion vector for eachmacroblock of a plurality of macroblocks; generating a refined motionvector for each macroblock of the plurality of macroblocks, based on themotion search motion vector; generating a direct mode motion vector foreach macroblock of the plurality of macroblocks, based on a plurality ofmacroblocks that neighbor the macroblock of pixels; generating a bestintra prediction mode for each macroblock of the plurality ofmacroblocks; determining a final motion vector for each macroblock ofthe plurality of macroblocks based on costs associated with the refinedmotion vector, the direct mode motion vector, and the best intraprediction mode; generating residual pixel values corresponding to thefinal motion vector for each macroblock of the plurality of macroblocks;and generating and storing neighbor data that includes motion vectordata associated one macroblock of the plurality of macroblocks in aplurality of data structures associated with each of a correspondingplurality of neighboring macroblocks for retrieval by at least one ofthe steps of generating a motion search motion vector, generating arefined motion vector, and generating a direct mode motion vector, whenoperating on at least one neighboring macroblock of the plurality ofmacroblocks to form a temporal prediction.
 15. The method of claim 14wherein at least one of the steps of generating a motion search motionvector, generating a refined motion vector, generating a direct modemotion vector, and generating a best intra prediction mode, operate in amacroblock adaptive frame and field mode and analyze each macroblock ofa plurality of macroblocks based on macroblock pairs that include a topframe macroblock and bottom frame macroblock from a frame of the videoinput signal and a top field macroblock and a bottom field macroblockfrom a corresponding field of the video input signal.
 16. The method ofclaim 14 wherein neighbor data includes frame below neighbor data forretrieval by a neighboring macroblock in a row below the one macroblockwhen processing in frame mode and field below neighbor data forretrieval by the neighboring macroblock in a row below the onemacroblock when processing in field mode.
 17. The method of claim 14wherein neighbor data includes frame right neighbor data for retrievalby a neighboring macroblock to the right of the one macroblock whenprocessing in frame mode and field right neighbor data for retrieval bythe neighboring macroblock to the right of the one macroblock whenprocessing in field mode.
 18. The method of claim 14 wherein at leastone of the steps of generating a motion search motion vector andgenerating a refined motion vector, generate at least one predictedmotion vector for each macroblock of the plurality of macroblocks usingretrieved neighbor data.
 19. The method of claim 14 wherein the step ofgenerating a direct mode motion vector generates at least one directmode motion vector for each macroblock of the plurality of macroblocksusing retrieved neighbor data.
 20. The method of claim 14 wherein thestep of generating a best intra prediction mode generates the best intraprediction mode for each macroblock of the plurality of macroblocksusing retrieved neighbor data.